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Evaluation of the RE system on the AIMED corpus 

AIMed corpus 

The AIMed corpus is widely used for PPI extraction 
(ftp://ftp.cs.utexas.edu/pub/mooney/bio-data/). The Aimed corpus consists of 225 
MEDLINE abstracts, of which 200 abstracts describe interactions between human 
proteins and 25 do not refer to any interaction. There are 5625 annotated sentences, 
1008 containing a true PPI (TRUE) and 4617 not containing a true PPI (FALSE). 

Evaluation of Kernel based Relation Extraction 

The performance of each model for association classification was evaluated by 
sentence-level 10-fold cross validation in each corpus. The classifiers' performances 
were assessed using P, R and F-score over the class TRUE. TRUE sentences contain 
real relationship between the entities analysed, in contrast with FALSE sentences where 
the two entities co-occur, but there is no semantic relationship between them. 
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Results 

The first experiments were conducted on the AIMed corpus in order to evaluate the 
performance of the Kdep kernel with different features and compare it with previous 
results (Table 1). Compared to the performance obtained with the original system {Ksl 
with sparse bigrams, P 52.9%, R 65.3%, F 58.1 %), Kdep alone does not show an 
improvement in performance (26: P 41.3%, R 61.2%, F 49%). However, the 
combination of Ksl and Kdep kernels results in an improvement of the results both at the 
level of precision and recall. For example, using role, lemma and stem as features leads 
to high levels of F-score (96: P 55.5%, R 67.2%, F 60.3%; 98: P 55.3%, R 67.3%, F 
60.3%). Although we show experiments where the features on the token for the v-walk 
and e-walk are tested one at a time, we also conducted experiments testing different 
combinations of these features, but the results were not better than the ones obtained 
with single features over the token (results not shown). Thus, the use of shallow 
linguistic information but also syntactic information in the form of dependency walk 
features lead to more accurate models for PPI relation extraction. Our results are 
comparable to the results obtained with state-of-the-art approaches tested on the 
AIMED corpus [1]. 
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Table 1: Selected results obtained by 10-fold cross-validation on the AIMed corpus. 

The best results achieved are highlighted in bold. The first column indicates the number of the 
experiment, the second column shows if K SL is used with (SB)/without (O) sparse bigrams, or if 
it is not used (-). The next two columns focus on K DEP features, that can be represented as v- 
walk and/or e-walk: token (T), stem (S), lemma (L), POS-tag (P), role (R) or none (-). The last 
columns show the result obtained in each experiment indicating precision (P), recall (R) and f- 
measure (Fl) in percentage (%). 
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